Skip to content

feat(storage,entity): path identity + speculation path→build mapping store#331

Open
behinddwalls wants to merge 1 commit into
preetam/ext/speculation-prioritizerfrom
preetam/int/storage-build-lookup
Open

feat(storage,entity): path identity + speculation path→build mapping store#331
behinddwalls wants to merge 1 commit into
preetam/ext/speculation-prioritizerfrom
preetam/int/storage-build-lookup

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

Integrating speculation needs the controllers to answer "which build belongs to this path" in both directions, without ever looking a row up by non-key attribute — the storage contract stays get/put-by-key so any KV backend can satisfy it. The build system mints its own build identifiers, and those stay the build store's primary key: the runner's ID is the natural name for the build row. What's missing is a durable link between a tree path and its build that exists independent of any in-flight message, plus a well-defined path identity to hang that link on.

What?

entity.SpeculationPathInfo gains ID — an assigned, immutable path identity, unique within its tree, minted by the speculate controller when it creates the tree (so it exists before any build is triggered and survives crashes). entity.SpeculationPathBuild is the new mapping entity ({PathID, BuildID, BatchID, Version, CreatedAt}, named after its speculation_path_build table) with a SpeculationPathBuildStore (Create/Get keyed by PathID): the forward path→build lookup, written only by the build controller, at most one build per path with ErrAlreadyExists making the existing row the truth on races. BatchID makes the row self-describing without parsing PathID's format; Version follows the repo's optimistic-locking convention (write-once today, reserved for future conditional re-pointing flows). entity.Build keeps the runner-minted ID as its primary key and gains SpeculationPathID as a plain column — the reverse build→path lookup; the previously embedded SpeculationPath value is dropped as a redundant denormalized copy of what the tree already stores. SpeculationPath.Equal (order-sensitive Base + Head) provides structural path identity for decision matching. entity.QueueID with ToBytes/QueueIDFromBytes mirrors the BatchID payload pattern for queue-scoped stages.

MySQL gains the speculation_path_build table and the build table swaps speculation_path/runner_id for speculation_path_id; schema files are picked up automatically (the schema dir is globbed by both Bazel and the testutil ApplySchema helper).

Test Plan

make gazelle && make fmt && make test (74/74). ✅ bazel test //test/integration/submitqueue/extension/storage/... against real Docker MySQL: Build round-trip with runner-style ID + SpeculationPathID, SpeculationPathBuild Create/Get round-trip incl. BatchID and Version, duplicate Create → ErrAlreadyExists, missing Get → ErrNotFound. Unit tables cover SpeculationPath.Equal and entity serialization incl. the new fields.

Issues

Stack

  1. feat(speculation): add enumerator + dependency-limit extensions #315
  2. feat(speculation): add path scorer extension #316
  3. feat(speculation): add selector + selection-limit extensions #317
  4. feat(speculation): add prioritizer + prioritization-limit extensions #320
  5. @ feat(storage,entity): path identity + speculation path→build mapping store #331
  6. feat(speculation): add parity default impls for the speculation seams #332
  7. feat(speculation): add probability path scorer #333

…store

## Summary

### Why?

Integrating speculation needs the controllers to answer "which build belongs to this path" in both directions, without ever looking a row up by non-key attribute — the storage contract stays get/put-by-key so any KV backend can satisfy it. The build system mints its own build identifiers, and those stay the build store's primary key: the runner's ID is the natural name for the build row. What's missing is a durable link between a tree path and its build that exists independent of any in-flight message, plus a well-defined path identity to hang that link on.

### What?

`entity.SpeculationPathInfo` gains `ID` — an assigned, immutable path identity, unique within its tree, minted by the speculate controller when it creates the tree (so it exists before any build is triggered and survives crashes). `entity.SpeculationPathBuild` is the new mapping entity ({PathID, BuildID, BatchID, Version, CreatedAt}, named after its `speculation_path_build` table) with a `SpeculationPathBuildStore` (Create/Get keyed by PathID): the forward path→build lookup, written only by the build controller, at most one build per path with ErrAlreadyExists making the existing row the truth on races. BatchID makes the row self-describing without parsing PathID's format; Version follows the repo's optimistic-locking convention (write-once today, reserved for future conditional re-pointing flows). `entity.Build` keeps the runner-minted `ID` as its primary key and gains `SpeculationPathID` as a plain column — the reverse build→path lookup; the previously embedded `SpeculationPath` value is dropped as a redundant denormalized copy of what the tree already stores. `SpeculationPath.Equal` (order-sensitive Base + Head) provides structural path identity for decision matching. `entity.QueueID` with ToBytes/QueueIDFromBytes mirrors the BatchID payload pattern for queue-scoped stages.

MySQL gains the `speculation_path_build` table and the build table swaps `speculation_path`/`runner_id` for `speculation_path_id`; schema files are picked up automatically (the schema dir is globbed by both Bazel and the testutil ApplySchema helper).

## Test Plan

✅ `make gazelle && make fmt && make test` (74/74). ✅ `bazel test //test/integration/submitqueue/extension/storage/...` against real Docker MySQL: Build round-trip with runner-style ID + SpeculationPathID, SpeculationPathBuild Create/Get round-trip incl. BatchID and Version, duplicate Create → ErrAlreadyExists, missing Get → ErrNotFound. Unit tables cover SpeculationPath.Equal and entity serialization incl. the new fields.
@behinddwalls behinddwalls force-pushed the preetam/ext/speculation-prioritizer branch from f3c68e1 to 0426ccb Compare July 10, 2026 19:24
@behinddwalls behinddwalls force-pushed the preetam/int/storage-build-lookup branch from 2551cf4 to bbc4956 Compare July 10, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant